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ABSTRACT 

This report examines trends in student learning. Using data 
from the National Assessment of Educational Progress, the first section 
reports on current trends in reading and mathematics test scores, 
highlighting arithmetic and noting that student’s computation skills have 
stagnated or declined in recent years. The second section revisits the 
previous year's study of high school culture. It replicates the 2001 survey 
of foreign exchange students with American students who have studied in high 
schools abroad, asking them to compare U.S. high schools to high schools in 
thirty-five countries. The American students reaffirmed key impressions of 
students from abroad, especially on the question of academic rigor; more than 
half of American students said their regular classes were easier than the 
classes they attended in foreign countries. By compelling margins, U.S. and 
foreign students agree that success at sports means much more to U.S. teens 
than to teens in other countries. However, a study of high schools that are 
sports powerhouses finds that there is no evidence that schools suffer 
academically when they excel in athletics. The third section discusses 
charter schools, examining the test scores of charters from 1999 to 2001 in 
10 states. Charter schools performed about one-quarter standard deviation 
below comparable regular public schools on these 3 years of state tests. The 
study did not indicate why they performed at this level. The report offers 
suggestions on how achievement in charters can be evaluated as fairly and 
accurately as possible. (Contains 37 endnotes.) (SM) 
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THE 2002 BROWN CENTER REPORT 
ON AMERICAN EDUCATION 

As in the past, the report is divided into three independent sections. 

The first section reports on current trends in test scores in reading and 
mathematics. Arithmetic receives special consideration. A troubling 
body of evidence is presented that suggests students’ computation 
skills have stagnated or even declined in recent years. Remarkably, the 
National Assessment of Educational Progress (NAEP), the nation’s report 
card, does not report how well elementary grade students are perform- 
ing in arithmetic. 

The second section of the report revisits last year’s study of high 
school culture. First, we replicate the 2001 survey of foreign exchange 
students with American students who have studied in high schools 
abroad, asking them also to compare U.S. high schools to high schools 
around the world. By compelling margins, American and foreign students 
agree that success at sports means much more to U.S. teenagers than 
to teens in other countries. Is this cause for concern? Does it interfere 
with the nation’s efforts to raise academic achievement? 

If holding athletic accomplishments in high esteem creates problems, 
one would expect them to surface in high schools with highly successful 
athletic teams. We present a study of high schools that are sports power- 
houses, schools that in recent years have been the best in the nation in 
football, baseball, and basketball. It is clear that these schools are excel- 
lent at sports. What about academics? Is dominance in team sports 
attained at a cost to excellence in reading and mathematics? After analyz- 
ing test score data from dozens of states, Brown Center researchers are 
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confident that the answer is no, excellence is not zero sum when it 
comes to sports and academics. There is no evidence that schools suffer 
academically when they excel in athletics. On state tests, the sports 
powerhouses score about as one would expect, no better or worse than 
non-powerhouse schools serving similar populations. And there is evi- 
dence, though only suggestive, that some schools are capable of making 
excellence at sports and excellence at academics mutually reinforcing. 

The third section of the report looks at charter schools. Charters 
have proliferated across the country since the first few opened in 
Minnesota nearly a decade ago. There are now about 2,400 charters 
serving 250,000 students. Very little is known about academic achieve- 
ment in charter schools, so we examined the test scores of charters from 
1999 to 2001 in ten states. In a nutshell, charter schools performed 
about one-quarter standard deviation below comparable regular public 
schools on these three years of state tests. We do not know why charters 
performed at this level. They may have attracted students who were 
already low achieving, which explains why parents sought an alternative 
to the local public school. Thus, readers are cautioned that these test 
scores may or may not reflect the quality of education students have 
received and are receiving at charters. And we offer a few suggestions 
on how achievement in charters can be evaluated as fairly and accurately 
as possible in the future, especially with state accountability systems 
beginning to take hold. 
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Part l The Nations Achievement 



T he brown center report annually examines 

trends in reading and math test scores. One source of data is the 
federal government’s National Assessment of Educational Progress 
(NAEP), also known as “the nation’s report card.” No new NAEP scores were 
released in reading and math during the past year. 



Data are also collected from state assessments. 
We obtained 2001 test scores for four different 
grades in forty-five states and analyzed whether 
scores had gone up, down, or remained the 
same from the previous school year. We then 
compared these year-to-year results to the 
year-to-year results of previous years. 

The percentage of states reporting annual 
gains in reading from 1999 to 2001 is displayed 
in Figure 1 . Compared to 2000, a few more 
states reported positive results for 2001 in 
eighth grade. But scores fell in grades 4, 5 , and 
10, extending declines already in place. In the 
past three years, the overall trend in reading 
is down. In all grades, fewer states reported 
higher reading scores in 2001 than in 1999. 

As mentioned in last years report, a plateau 
effect is quite common in scores after a partic- 
ular test is given for a while. Typically, strong 
gains are recorded in the early years of a tests 
use, then achievement flattens out or declines. 
Many states began annual testing of students 
in the late 1990s, and their assessment pro- 
grams have reached maturity. Nevertheless, 
the states are indicating that reading achieve- 
ment may have stagnated. 1 



ERIC 



z Brown Center Report on American Education 



A similar pattern is evident in math 
(see Figure 2). In 2000, state tests gave encour- 
aging signals about math achievement in 
grades 8 and 10, but the gains dissipated in 
2001. In grades 4 and 5, about 50% of states 
reported improved math scores in 2001 . This 
is down from 60% in 2000 and about 80% 
in 1999. The eighth grade experienced the 
biggest decline in 2001, with fewer than 40% 
of states reporting gains. The trends in grades 
4 through 8 warrant national attention and 
concern. The elementary grades are crucial 
for learning arithmetic and, in particular, 
computation skills. 

Turning our backs on arithmetic 

Dictionaries define arithmetic as the most 
elementary branch of mathematics. As such, 
it is the foundation on which the field of 
mathematics rests and a universal starting 
point for learning math. For young children, 
arithmetic is to mathematics as phonics is 
to reading or as learning about the colonies 
and the Civil War is to learning our nations 
history. Students who do not learn arithmetic 
are not prepared to learn algebra or calculus. 

9 



Reading gains 
continue to slow. 
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Introduction to NAEP 



Why is it that the 
NAEP, the federal 
government’s primary 
tool for evaluating 
American education, 
cannot tell whether 
fourth graders know 
how to compute 
accurately? 



They are less likely to attend college. When 
they enter the workforce, their chances 
of landing middle class jobs are severely 
diminished. 2 

Computation skills are central to arith- 
metic. The term “computation skill s” refers 
to the ability to add, subtract, multiply, and 
divide whole numbers and to perform these 
same operations using fractions, decimals, 
and percentages. Their importance makes 
the following fact truly alarming: the NAEP 
test does not report student progress in 
computation. Not even at fourth grade. The 
NAEP reporting categories for fourth grade 
are Number Properties and Operations, 
Geometry, Data Analysis, Algebra and Func- 
tions, and Measurement. Computation skills 
are subsumed under the category Number 
Properties and Operations, not reported 
separately. And that category comprises only 
40% of the fourth grade math test. Beginning 
in 2005, the category will shrink from the 
current 25% to 20% of the eighth grade test. 
Granted, items requiring computation are 
scattered across other categories of the NAEP — 

O 
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a geometry item may require students to 
compute the area of a triangle, for example — 
but they are never linked together to gauge 
computational proficiency. 

Here is what the 2005 NAEP Mathe- 
matics Framework says: 

“It is important to note that there are 
certain aspects of mathematics that occur 
in all the content areas. The best example 
of this is computation. Computation is the 
skill of performing operations on numbers. 
It should not be confused with the content 
area of NAEP called Number Properties 
and Operations, which encompasses a 
wide range of concepts about our numera- 
tion system. . . .Certainly the area of Number 
Properties and Operations includes a vari- 
ety of computational skills, ranging from 
operations with whole numbers to work 
with decimals and fractions and finally 
real numbers. But computation is also crit- 
ical in Measurement and Geometry, such 
as in calculating the perimeter of a rectan- 
gle, estimating the height of a building, 

U 




. . no single instrument of youthful education has such mighty power, 
both as regards domestic economy and politics, and in the arts, as the 
study of arithmetic. Above all, arithmetic stirs up him who is by nature 
sleepy and dull, and makes him quick to learn, retentive, shrewd, and 
aided by art divine he makes progress quite beyond his natural 
powers. ” Plato, Laws, Book V (360 B.C.) 



or finding the hypotenuse of a right triangle. 
Data analysis often involves computation, 
such as calculating a mean or the range of 
a set of data. Probability often entails work 
with rational numbers. Solving algebraic 
equations usually involves numerical 
computation as well. Computation, there- 
fore, is a foundational skill in every content 
area. While the main NAEP assessment is 
not designed to report a separate score for 
computation, results from the long-term 
NAEP assessment can provide insight into 
students’ computational abilities.” 3 

As applied to elementary mathematics, the 
term “Number Properties and Operations” is 
classic educational jargon. It sounds sophisti- 
cated but is ambiguous. It is also of marginal 
importance. Most people, including parents, 
are justifiably concerned that children learn 
how to add, subtract, multiply, and divide 
whole numbers and accurately use fractions, 
decimals, and percentages. Number Properties 
and Operations? What parent is worried about 
that? Why is it that the NAEP, the federal gov- 
ernment’s primary tool for evaluating American 
education, cannot tell whether fourth graders 
knowhow to compute accurately? 

How did this happen? 

In 1989, the National Council of Teachers of 
Mathematics (NCTM) released the Curriculum 
and Evaluation Standards for School Mathematics. 
The standards proposed that the K-12 math 

BESTCOPY AVAILABLE J2 



curriculum be organized into five strands. 
Federal officials at the National Assessment 
Governing Board (NAGB), the group in 
charge of the NAEP, agreed. The NAEP 
reporting categories listed above are the 
NCTMs five strands. Arithmetic was rele- 
gated to a subcategory of number because 
the NCTM standards argued that “shop- 
keeper arithmetic” commanded too much 
time in elementary classrooms. For most 
of the twentieth century, arithmetic had 
dominated the K-8 math curriculum, espe- 
cially the teaching of computation skills. 

The NCTM felt it was time for a change. 
Calculators would free students from the 
drudgery of memorizing multiplication 
tables and practicing long division. Rather 
than learning standard algorithms through 
direct instruction, arithmetic could be 
learned while solving “real world” problems 
that piqued children’s interest. The federal 
government — from NAGB to the National 
Science Foundation to the Department of 
Education — enthusiastically embraced 
this position. Unfortunately, official support 
preceded any practical experience with 
the NCTM standards or independent 
research on their effects. Potential conse- 
quences were unknown. So we arrived 
where we are today: a federally endorsed 
state of ignorance on the computation skills 
of American students. 

Are computation skills 
improving or declining ? 

In order to assess student progress in com- 
putation, evidence must be pulled together 
from several disparate sources, including 
the NAEP As noted in the quotation from 
the NAEP math framework, the long term 
trend NAEP can help. Indeed, as pointed 
out in previous editions of The Brown Center 
Report , the two NAEP tests have been signal- 
ing a potential problem for some time 
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(see Figures 3 and 4). Scores on the main 
NAEP, a test reflecting the NCTM’s views, sky- 
rocketed during the 1990s. Astonishingly, the 
gains reflected in Figure 3 are the equivalent 
of more than a full school year’s worth of 
learning. Scores on the trend NAEP, on the 
other hand, have bogged down. Granted, 
the divergence of the two tests could be due 
to a number of reasons. The main NAEP 
allows students to use calculators on one- 
third of the test and asks questions in which 
partial credit may be granted. One of the 
most plausible explanations, however, is 
the prominence of computation items on 
the trend test and their subordinate role on 
the main test. 

NAEP officials release performance on 
individual items of the trend test. Examining 
the scores since 1982 on arithmetic items is 
quite revealing (see Figure 5). Bear in mind 
when studying Figure 5 that the performance 



of different age groups cannot be compared. 
Figure 5 is not suggesting that thirteen year 
olds scored higher than seventeen year 
olds. The different age groups take different 
tests, but for each group, test questions 
have remained essentially the same over time. 
The data are plotted on the same graph to 
identify points in time when achievement 
changed direction. Solid gains were registered 
in the 1980s for all three age groups. Then 
something happened around 1990. Scores 
were flat for nine and thirteen year olds after 
1990, and they declined sharply for seven- 
teen year olds. In 1999, a smaller proportion 
of seventeen year olds had mastered basic 
arithmetic than a decade earlier. The headline 
story here is that scores were up before 1990 
and flat or down afterwards. 4 

The decline of seventeen year olds’ 
performance is largely attributable to sharply 
falling scores on items involving fractions 



The main NAEP shows 
dramatic gains in math. 



From 1990 to 2000, fourth 
grade math scores increased 
fifteen points — equivalent to 
about 1.2 years of learning. 
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The trend NAEP 
is essentially flat. 
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(see Figure 6). Thirteen year olds’ scores also 
slipped in the 1990s, but not by as much as 
the older group. Nine year olds are not tested 
on fractions. Only five items make up the 
fractions cluster at age seventeen and four 
items at age thirteen so not too much should 
be made of the patterns detected here. But 
they do warrant concern. As Figure 6 dis- 
plays, the percentage of seventeen year olds 
correctly answering items with fractions 
fell steadily — 61% in 1990, 58% in 1992, 
56% in 1994, 53% in 1996, 48% in 1999. 
Statistical tests confirm that the decline in 
seventeen year olds’ performance — displayed 
in both Figures 5 and 6 — was significant 
from 1990 to 1999. 5 

School districts in the state of Iowa 
have been giving the same achievement test 
for several decades — the Iowa Test of Basic 
Skills (ITBS). No other state has collected 
comparable achievement data over such an 



extended period of time. The math portion 
of the ITBS contains a computation subtest, 
allowing for separate scrutiny of the paper 
and pencil skills that came under attack by 
math reformers in the late 1980s. Figure 7 
displays test scores for Iowa’s eighth graders 
from 1978 to 2001. Eighth grade is crucial 
in any discussion of arithmetic, for it is when 
most students begin the transition from 
arithmetic to algebra. 6 

Iowa’s scores on computation and 
non-computation subtests rose together in 
the 1980s, increasing by about one-half grade 
level. Computation flattened out from 1988 
to 199 1 , while non-computation scores contin- 
ued rising. Then in 1992, computation scores 
went into a swan dive for several years, losing 
more than one-half grade level by 200 1 . All 
of the 1980s gains were erased, and computa- 
tion skills hit low levels not seen in more than 
two decades. Although it is true that Iowa’s 



When it comes to 
computation skills, 

Iowa may be the canary 
in the coal mine, 
warning the nation that 
there are consequences 
to de-emphasizing 
computation skills in 
the elementary grades. 
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The nation must not 
remain in the dark on 
whether students are 
learning arithmetic. 



schools experienced demographic changes 
in the 1990s, there is no reason to think that 
these changes would affect computation skills 
differently than other math skills. 7 When it 
comes to computation skills, Iowa may be the 
canary in the coal mine, warning the nation 
that there are consequences to de-emphasizing 
computation skills in the elementary grades. 8 

Conclusion 

Let’s put the computation issue in perspec- 
tive. The widespread math reforms of the 
1990s may have forced a trade-off. Since 
1990, U.S. students have registered gains in 
several math areas, especially problem solv- 
ing, geometry, and data analysis. These gains 
have been documented in previous Brown 
Center reports. 9 But computation skills have 
been flat at best, and there is some evidence 
that they have declined. In 1999 on the trend 
NAEP, a smaller percentage of thirteen and 



seventeen year olds answered items with 
fractions correctly than a decade earlier. 
Seventeen year olds’ scores on arithmetic 
items fell significantly from 1990 to 1999. 
In 2001 , eighth grade computation scores 
on the ITBS hit twenty-three year lows in 
the state of Iowa. 

This evidence is not conclusive, but it 
does suggest that continuing to ignore com- 
putation would be a mistake. Three policy 
recommendations flow from the analysis. 

• The nation must not remain in the dark 
on whether students are learning arith- 
metic. It is especially important that 
arithmetic is assessed on the NAEP at 
the fourth and eighth grades. This score 
should be reported separately from other 
categories and should not include items 
on which calculators are used. If math 
reform involves a trade-off in students’ 
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math skills, we must carefully measure 
what is being lost and gained. Only 
then can the wisdom of current practices 
be determined. 

• The NAEP is only valuable to parents 
and teachers if it reports student progress 
on skills that are important to them. 
Subcomponents of arithmetic should be 
assessed and reported separately, includ- 
ing operations with whole numbers at 
the fourth grade and operations with 
fractions, decimals, and percentages at 
the eighth grade. 

• The NAEP test should establish indepen- 
dence from the NCTM’s reform agenda in 
order to objectively gather the evidence 
on which NCTM’s reforms will be judged. 
The current framework and the frame- 



“Mathematics is the queen of the sciences and arithmetic the queen of 
mathematics. She often condescends to render service to astronomy and 
other natural sciences, but in all relations she is entitled to the first rank. ” 
Friedrich Gauss, quoted by R.E. Moritz, Memorabilia Mathematica (1914) 



work going into effect in 2005 are based 
on NCTM doctrine. 

• Arithmetic deserves the same attention 
that reading has received in federal 
education policy. The rising math scores 
on the main NAEP may have lulled the 
nation into thinking all is well in young 
people’s math skills. A national cam- 
paign emphasizing arithmetic and 
computation skills would help prepare 
all students for advanced math courses 
in high school. 
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Part II High School Culture 



I NTERNATIONAL COMPARISONS OF LEARNING IN 
the elementary grades cast U.S. schools in a favorable light. In the 
fourth grade, American students rank among the top one-third of 
nations in mathematics. In reading, the U.S. performs even better, one of 
three or four countries with the highest test scores. High school compar- 
isons, on the other hand, are a national embarrassment. In math and 
science, U.S. students score well below average. 



Why do older students do so poorly? Last 
year’s Brown Center Report featured a survey 
of foreign exchange students who had 
recently attended U.S. high schools. We asked 
them to compare high schools in their home 
countries with American schools on several 
dimensions, including rigor of curriculum, 
frequency of homework, what motivates 
teens to attend school, and the value students 
place on success at sports and mathematics. 
The purpose was to see what teenagers 
from other countries think about important 
aspects of U.S. high schools and American 
teen culture. 



year. 10 We received responses from 328 
students (58%) who had attended high 
schools in thirty-five host countries. Some of 
the results are discussed here, and responses 
to all of the survey’s questions can be found 
on our website (www.brookings/browncen- 
ter) 1 1 . Readers are cautioned that foreign 
exchange students — from the U.S. or any 
other country — are certainly not representa- 
tive of all students in a particular country. 
The findings can only be generalized to 
students in the AFS program. Despite this 
limitation, the study is valuable in offering a 
glimpse into the life of high schools and the 
values of teen culture from the perspective 
of teens from all over the world. Rarely are 
Americans allowed to view their institutions 
through the eyes of others. Rarer still are 
cross-national comparisons of high schools 
by the students who attend them. 



In the summer and fall of 200 1 , we 
replicated the survey with what seems to be a 
natural comparison group, American students 
who have attended high schools abroad in 
the same exchange program. Surveys were 
mailed to the 562 students who had partici- 
pated in the AFS Intercultural Exchange 
Program during the 2000-2001 academic 



The American students reaffirm the 
key impressions of students from abroad, 
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especially on the question of academic rigor 
(see Figure 8, page 17). U.S. high schools 
are not very demanding. For example, 85% 
of students from abroad found classes in 
American high schools easier than classes at 
home (56% much easier, 29% a little easier). 
American students agree, although not as 
emphatically. More than half, 56%, say their 
regular classes in the U.S. are much easier or a 
little easier than the classes they attended in 
foreign countries. About 30% say U.S. classes 
are harder, almost three times the percentage 
of foreign students that felt that way. 

American students also agree that 
students in the U.S. spend less time on 
schoolwork than students in other nations 
(see Figure 9, page 17). Students from other 
countries were divided about two to one on 
this question, with 57% stating that American 
students spend less time on schoolwork and 
25% saying more time. Among U.S. students, 



54% feel that American students spend less 
time on their studies and 34% say more time. 
American high schools are seen as less focused 
on academic learning than high schools of 
other nations, and this perception is shared 
by both American and foreign students. 

American students encounter two 
distractions in high school that other nations 
minimize, part-time work and sports. The 
U.S is unique in the world when it comes to 
employment during high school. Most coun- 
tries discourage it. For American students, 
holding part-time jobs is the norm. In our 
survey, more than half of U.S. students were 
employed during the school year, but only 
about one-fourth of students from other 
countries (see Figure 10). About one-third of 
American students (35%) reported that they 
worked more than five hours per week, com- 
pared to only 9% of students from foreign 
countries. The survey question specifically 



American students 
encounter two 
distractions in high 
school that other 
nations minimize, 
part-time work 
and sports. 



U.S. students are more likely to work at part-time 
jobs during a typical school week. 




no time 
45 % 



>5 

35 % 



10 




Margin of error: +/- 6% 
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Both groups agree that math 
is valued less by U.S. students. 



11 



International students 



U.S. students 



International and U.S. 
students were asked: 
Compared to students 
in other countries, how 
important is it to your 
American friends to do 
well in math? 



about 
the same 
41 % 



a little 
more 
10 % , 



much 

more 

4 % 



much 

less 

18 % 



a little 
less 
27 % 




Margin of error: +/- 



Both groups resoundingly agree that 
the U.S. students highly value sports. 



International students 



U.S. students 
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The U.S. permits sports 
a place in the life of 
high schools that other 
countries avoid. 



asked for the number of hours spent working 
before and after school. It appears that even 
on school days a significant portion of U.S. 
students’ out-of-school time is devoted to 
employment. 

Part-time work may capture students’ 
time. Sports captures their hearts. One of 
the most striking results of last year’s survey 
was the contrast foreign exchange students 
perceived between how much American high 
school students value success at sports com- 
pared to success at mathematics. American 
students see a similar contrast. We asked 
the American exchange students to compare 
pursuits in which U.S. students and students 
abroad want to do well. Mathematics was 
the first topic. About 37% said American 
students value success at math much less 
or a little less than teens in other countries, 
38% said about the same, and 25% said 
much more or a little more (see Figure 1 1 , 
page 19). According to the U.S. students, 
mathematical prowess is slightly more 
esteemed abroad, but not by much. 

Sports is quite a different story With 
both samples, the survey responses were 
overwhelming (see Figure 12, page 19). 
Two-thirds of both groups say that American 
students care much more about athletic 
accomplishments than students in other 



countries, swamping the 4% to 6% who say 
they matter more abroad. On first blush, 
this is not terribly surprising. For some time, 
sports have commanded a prominent place 
in American popular culture. From Babe 
Ruth to Muhammad Ali to Michael Jordan, 
America’s most beloved national heroes have 
included athletes. But the U.S. is not unique 
in that regard. Soccer is practically a national 
religion in some countries. Baseball is revered 
in Japan. And teens around the world are 
just as sports crazy as teens in the U.S. 

A significant difference is that the 
U.S. permits sports a place in the life of high 
schools that other countries avoid. Team 
sports abroad are often organized by clubs 
outside the school. Community pride in its 
local team may be just as great, but teams 
represent the communities themselves, not 
local high schools. Primarily American phe- 
nomena are the massive, costly high school 
stadiums and arenas in which sports are 
played, extensive press coverage of inter- 
scholastic competition, school wide rallies 
during the school day, extensive travel by 
student athletes, and high school coaches 
signing big dollar shoe contracts. Sports 
have a unique role in U.S. schools. What 
do we know about the effect of sports on 
American education? 
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N THE 1950s, THE EMINENT SOCIOLOGIST JAMES COLEMAN 
described the unparalleled position student athletes command in 
the social status of high schools. Athletes are revered. Good students, 
on the other hand, are frequently the outcasts of teenage society. Coleman 
argued that unless adults pay close attention to the values of teens — and 
do not shrink from guiding them — sports may adversely affect school 
culture by undermining the pursuit of academic goals. 



In the extreme, teens are capable of trans- 
forming personal qualities that adults admire 
into character weaknesses, cruelly so when 
detected in peers. Respect for authority, 
working hard, and intellectual brilliance are 
the defining characteristics of “suck-ups,” 
“grinds,” and “brains.” Not much has changed 
in this regard since the 1950s. Cornell 
economist John Bishop has documented the 
same phenomena in recent studies of high 
school “nerds.” 12 The cautionary advice of 
these studies deserves attention. Schools func- 
tion like mini-societies. A high schools culture 
represents the shared values of the institution, 
and it shapes student behaviors as much as 
it reflects them. 

What about the athletes themselves? 

Do sports help or hinder their learning? In 
general, the research is strongly positive on 
participating in high school sports. Herbert W 
Marsh analyzed survey data collected from a 
large random sample of high school students 



in the 1980s. He found participating in sports 
has a positive influence on fourteen of twenty- 
two student outcomes, including enrollment 
in academic coursework, homework, and 
reduced absenteeism. 13 Other well-designed 
studies show benefits too. Studies of high 
school students in the early 1990s found 
a positive effect of participating in athletics 
on both grades and test scores. 14 Studies 
also document the positive impact of sports 
on several aspects of students’ psychological 
and social development, such as boosting 
students’ self-concepts, reducing delinquency 
and discipline problems, and diminishing the 
chances of teen pregnancy in female athletes. 15 

High school athletes appear to reap 
benefits after graduation. Students who par- 
ticipate in sports hold higher educational 
aspirations while in high school, and, subse- 
quently, they are more likely to attend college. 
Economists have shown that high school 
athletes have an earnings advantage over 
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non-athletes ten to fifteen years after gradua- 
tion. 16 In particular, the adult wages of black 
males are higher if they participate in sports. 17 
Bradley T. Ewing estimates that African 
American adults who participated in high 
school sports earn 8% to 11% more than 
those who did not. 18 

There are several theories as to how 
and why the positive effects of sports occur. 
Some explanations focus on the development 
of what is known as “human capital.” For a 
moment, think of young people as small 
business firms. New businesses need capital 
investments to grow and mature — labor, 
financing, equipment. Children need the 
knowledge, habits, and skills that allow them 
to become productive adults. Start-up com- 
panies acquire capital from investors. Children 
acquire human capital from schools, but 
also through education in the broadest sense 
of the term: in the home, from peers, and 
in the community. From the human capital 
perspective, high school sports enhance an 
individual’s stock of productive resources. 
Athletes may learn self-discipline, how to 
follow directions, perseverance, and how 
to set goals, a valuable set of skills for success 
in college and the workplace. 

Another explanation focuses on social 
capital. High school sports promote strong 
ties between athletes and schools, bring par- 
ents of athletes into close contact, and create 
dense social networks around youngsters. A 
social network is a fancy term for something 
simple. Small towns possess dense social 
networks. The adults in an adolescent’s life 
know each other. If an athlete decides to do 
something stupid, it is likely that an adult 
will hear about it and have a chance to inter- 
vene. Non-athletes, especially at high schools 
with two or three thousand students, often 
pass through schools anonymously. Another 
facet of social capital relates to establishing 
valuable social connections. Athletes learn 
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teamwork, and by being around other moti- 
vated students, may make solid friendships 
that last long into adulthood. 

A third explanation involves signaling. 
Teachers, colleges, and employers believe 
that successful athletes possess attributes that 
are also common to good students and good 
employees. Kids who go out for a team sport 
may be intrinsically different than others — 
more ambitious, harder working, more 
confident in themselves. Signaled of the like- 
lihood that a person possesses these traits, 
educators and employers reward athletes 
with good grades, admission to college, good 
jobs, and high wages. Notice the difference 
between the human capital and social capital 
explanations, on the one hand, and the sig- 
naling explanation, on the other. The human 
and social capital stories are that athletes 
benefit from participation in sports because 
playing sports adds value, producing better 
students and better employees. The signaling 
story is that educators and employers believe 
anyone who self-selects into sports — whether 
the participation adds value or not — probably 
possesses valuable characteristics. 

There are apparent limits to the pay-off 
from sports. In a large study of high schools, 
Laurence Steinberg discovered that students 
who devote more than twenty hours per week 
to extracurricular activities, including sports, 
suffer academic losses. Spending an inordi- 
nate amount of time on sports or allowing 
athletics to circumvent studying can turn the 
benefit of athletic participation into a loss. 

In addition, the positive effects of high school 
sports may not be uniform for all sports or all 
students. A recent study, for example, found 
that participation in basketball or football 
had a negative effect on tests scores, but was 
neutral on grades. Other sports, primarily, 
baseball and track, had a positive effect on 
white students’ grades, but a negative effect 
on the grades of African Americans. 19 Race 




Kids who go out for 
a team sport may he 
intrinsically different 
than others — more 
ambitious, harder 
working, more confident 
in themselves. 
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differences have also been discovered in how 
participation in high school sports is related 
to adult earnings. A 2001 study uncovered 
a positive impact on blacks’ earnings, but a 
negative impact on whites. 20 Only recently 
have researchers accessed databases large 
enough to dig down to analysis at this level, 
so more research is needed to reach any sub- 
stantive conclusions. 

Returning to Coleman’s work high- 
lights an additional limitation. Most of the 
/ research cited above focuses on student 
athletes. What about other students? What 
about school culture as a whole? The starting 
five basketball players in a school with 1,500 
students constitute only one-third of one 
percent of all students at the school. The lit- 
erature is fairly convincing that they derive 
benefits from being athletes. But if a focus 
on excellence in sports has an adverse effect 
on school culture, then many more students 
might be negatively affected. Does this 
happen? A recent study in Massachusetts 
found that districts spending more on sports 
tend to score lower on the state’s achievement 
test. 21 But there is sparse research on the 
question, and the notion that sports can 
undermine a school’s academic performance 
remains largely theoretical. 

Standouts and powerhouses 
in high school sports 

How seriously do sports distract American 
high schools from their academic mission? 
We looked for an answer to this question 
by focusing on a sparsely researched segment 
of U.S. schools — high schools that are nation- 
ally recognized as top schools in major 
sports, so-called “standout” or “powerhouse” 
schools. If sports impede academic success, 
it should be noticeable in these schools. 
Identifying standouts took some work. We 
used Parade Magazine’s All-America Teams 
and USA Todays national and regional 



rankings to identify the top high schools in 
three team sports — football, baseball, and 
basketball. Rankings at the end of each sport’s 
season since the 1997-98 school year were 
collected and coded. In football, we were 
able to gather data back to the 1990-1991 
academic year. 

Bear in mind that this is only the first 
cut at identifying dominant schools. Indeed, 
the Parade list of all-star players is probably 
not an appropriate tool to screen for team 
dominance. The intention here, however, was 
first to conduct a sweep of high schools that 
were good enough to be noticed by national 
press, even if they had only one star player. 

So although the standout pool includes some 
schools that are not powerhouses, the approach 
identifies a large enough sample, so that 
national patterns of athletic excellence might 
be revealed. 

Where are the standout high schools 
located? Table 1 shows the top ten states in 
the three sports. The top ten states are defined 
by “over-representation” of standouts, that is, 
the amount that a state’s portion of standouts 
exceeds the amount one would expect, 
which is simply the state’s portion of the U.S. 
population. In other words, values in the last 
column of Table 1 were found by computing 
the difference between the two previous 
columns (with a few discrepancies due to 
rounding). In football, for example, Texas has 
forty-eight standout high schools, represent- 
ing 10.9% of the national total. Given that 
Texas holds about 7.5% of the nation’s popu- 
lation, its percentage of standout schools is 
3.4% points more than expected. 22 

The most notable aspect of Table 1 is 
the geographical distribution of schools in 
the three sports. Indeed, regional stereotypes 
from popular books and films are reinforced. 
Football is dominated by southern states. 

The top eight states are located in the South. 
This is not at all surprising if one reads Friday 
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Night Lights , the compelling story of the west 
Texas town of Odessa and its 1988 champi- 
onship football team from Permian High 
School. In basketball, three out of the top 
five states are in the Midwest. The passion 
for basketball in the Midwest, most notably 
Indiana, is movingly documented in William 
Gildea’s Where the Game Matters Most . The 
basketball standouts also seem to be states 
with notable state universities. In the top five 
states, the universities of Illinois, Michigan, 
Maryland, Indiana, and North Carolina pos- 
sess legendary basketball programs. The 
correlation is not perfect — Kentucky and 
Kansas are noticeably absent from the top 
states — but it’s certainly plausible that strong 
college sports programs influence the quality 
of high school athletics. Baseball has no clear 
pattern. The top three states — California, 
Florida, and Arizona — have obvious climate 
advantages for baseball, but Newjersey 
also makes the list. Excellence in baseball is 
less dispersed than the other two sports. 
California and Florida produce more than 
one-quarter of the nation’s standout schools 
in the sport. 23 

Another interesting aspect to 
the standout schools is not shown in the 
table: the large percentage of private schools. 
Approximately 19% of the standouts are 
private schools, much larger than the share 
of high schools nationally (about 11%). This 
could be tied to recruiting. Private schools 
are not constrained by residential boundaries 
and can recruit talented athletes to attend 
their schools. The award winning documen- 
tary Hoop Dreams told the story of two star 
basketball players growing up in central 
Chicago. One of the young men was heavily 
recruited to attend a suburban Catholic 
school, where he eventually enrolled. Private 
schools are sensitive to the charge that they 
aggressively recruit star athletes, so it is impor- 
tant to emphasize that the current study 
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Top Ten States in Three Sports 



Table 



(States ordered by percent of over-representation of "standouts") 



Football (N=442) 



State 


No. of Standouts 


% of Standouts 


% of U.S. Population 


% Over 


Texas 


48 


10.9 


7.5 


3.4 


Florida 


37 


8.4 


5.7 


2.6 


Mississippi 


13 


2.9 


1.0 


1.9 


Alabama 


15 


3.4 


1.6 


1.8 


South Carolina 


14 


3.2 


1.4 


1.7 


Georgia 


19 


4.3 


2.9 


1.4 


Louisiana 


11 


2.5 


1.6 


0.9 


Virginia 


14 


3.2 


2.5 


0.6 


Montana 


4 


0.9 


0.3 


0.6 


Oklahoma 


7 


1.6 


1.2 


0.3 



Basketball (N=29i) 



State 


No. of Standouts 


% of Standouts 


% of U.S. Population 


% Over 


Illinois 


19 


6.5 


4.5 


2.1 


Michigan 


16 


5.5 


3.6 


1.9 


Maryland 


11 


3.8 


1.9 


1.9 


Indiana 


11 


3.8 


2.2 


1.6 


North Carolina 


13 


4.5 


2.9 


1.6 


Tennessee 


9 


3.1 


2.0 


1.0 


Oklahoma 


6 


2.1 


1.2 


0.8 


Louisiana 


7 


2.4 


1.6 


0.8 


Iowa 


5 


1.7 


1.1 


0.7 


Washington 


8 


2.7 


2.1 


0.6 



Baseball (N=i7i) 



State 


No. of Standouts 


% of Standouts 


% of U.S. Population 


% Over 


Florida 


19 


11.0 


5.7 


5.2 


California 


28 


16.2 


12.2 


4.0 


Arizona 


7 


4.0 


1.8 


2.2 


New Jersey 


9 


5.2 


3.0 


2.2 


Washington 


6 


3.5 


2.1 


1.3 


Indiana 


6 


3.5 


2.2 


1.3 


Georgia 


7 


4.0 


2.9 


1.1 


Connecticut 


4 


2.3 


1.2 


1.1 


Louisiana 


4 


2.3 


1.6 


0.7 


West Virginia 


2 


1.2 


0.7 


0.5 



"Standouts" were named in the top 25 national rankings or top 10 regional rankings in USA Today or had a 
player named to the Parade All-American team. 
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How Big Are Powerhouse High Schools? 

(N=141) 



Table 



2 





< 1000 
Students 


1000-1499 

Students 


1500-1999 

Students 


>2000 

Students 


Median 


U.S. High Schools 


60% 


19% 


12% 


10% 


791 


Powerhouse 
High Schools 


10% 


19% 


25% 


46% 


1,920 


Football 


10% 


20% 


20% 


50% 


2,012 


Basketball 


15% 


23% 


26% 


j 36% 


1,744 


Baseball 


2% 


14% 


26% 


57% 


2,134 



Enrollment data from U.S. Department of Education, National Center for Education Statistics, Common Core 
of Data, “Public Elementary/Secondary School Universe Survey,” 1999-2000. 

“Powerhouses” were named in the top 25 national rankings of USA Today. 



In What Kinds of Communities Are 
Powerhouse High Schools Located? 

(N=141) 



Table 

3 





Urban 


Suburban 


Rural 


U.S. High Schools 


19% 


31% 


50% 


Powerhouse High Schools 


36% 


54% 


10% 


Football 


22% 


68% 


10% 


Basketball 


59% 


36% 


6% 


Baseball 


26% 


60% 


14% 



Data on communities from U.S. Department of Education, National Center for Education Statistics, Common 
Core of Data, “Public Elementary/Secondary School Universe Survey,” 1999-2000. 



cannot determine whether recruiting is 
taking place. Gifted athletes maybe attract- 
ed to private high schools for a variety of 
reasons. We only observe that private high 
schools produce more standouts in team 
sports than the proportion of private high 
schools overall. Among the standouts, 30% 
of basketball schools are private schools. 24 




Powerhouses: The best of 
the best 

In the second cut of the data, we identified 
163 powerhouses, schools ranked in the 
top twenty-five nationally in USA Todays 
end-of-season rankings since 1997. These 
are America’s dominant high schools in team 
sports. We limited the analysis to public 
schools because of the spotty data available 
on private schools, especially the lack of test 
scores. We were able to collect the following 
data on 141 schools (88% of powerhouses): 
enrollment, community (urban, suburban, 
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rural), racial composition, and poverty 
(percentage of students eligible for free 
lunch). 25 The powerhouses are located in 
twenty-four states. They have several charac- 
teristics that make them different from the 
average high school. 

Powerhouse schools are huge. 

The median powerhouse serves 1,920 
students, more than twice as many students 
as the median high school in the U.S. Only 
10% of American high schools serve more 
than 2,000 students, compared to 46% of 
the powerhouses (see Table 2). Large schools 
have a deep pool of talent from which to 
assemble athletic teams. Competition in 
baseball and football is strongly influenced 
by school size, as over half of the dominant 
high schools in these two sports are popu- 
lated by at least 2,000 students. Perhaps 
because of its smaller squads, basketball is 
not as driven by school size, but even it 
favors large schools. Small schools are at a 
marked disadvantage in team sports. In the 
U.S., 60% of high schools are attended by 
1,000 students or less. These schools have 
little chance of rising to prominence in team 
sports. Only 10% of the powerhouse high 
schools are that small. 

Different sports flourish in different 
kinds of communities. 

Basketball is an urban sport (see Table 3). 
More than half of the nation’s dominant bas- 
ketball teams hail from urban high schools. 
Football and baseball are dominated by sub- 
urban schools. The availability of open space 
for the large playing fields of baseball and 
football is probably one reason for the dis- 
crepancy. But the relative wealth of suburbs 
also might be a factor. Rural schools constitute 
about one-half of the nation’s high schools, 
but only 10% of the powerhouse schools in 
the three sports. Basketball’s appeal seems 
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to diminish as it gets farther from the city. 
Dominance in basketball is exceedingly rare 
for a rural high school, with only 6% of the pow- 
erhouses in that sport coming from rural areas. 

The South is the dominant region 
in high school sports. 

We used the categories of the National 
Assessment of Educational Progress to sort 
the powerhouses by geographical region 
(see Table 4). The South is extraordinary in 
high school sports. The region holds 43% of 
the powerhouse schools, more than double 
its 21% share of U.S. high schools. The South 
is especially strong in football and baseball. 
The Northeast is the most underrepresented 
region, with 1 1% of the powerhouses. The 
Midwest’s strongest sport is basketball. 

The West’s strongest sport is baseball, and 
as noted above, dominance in baseball is 
greatest in the sunbelt states of the South 
and West. 



Powerhouse schools are attended 
by a large African American population. 

Student enrollment in the average high 
school in the U.S. is 72% white, 13% African 
American, 10% Hispanic, and 3% Asian. 

The powerhouse high schools are attended 
by 28% of African American students, about 
twice the percentage of the average high 
school (see Table 5). Dominant schools in 
basketball and football serve substantial 
black populations. The dominant basketball 
schools are 42% African American and 40% 
white, and in football, 64% white and 26% 
African American. Baseball powerhouses are 
similar to national averages, with 73% white 
enrollment. The figures square with racial 
patterns in data collected from high school 
sophomores in 1990. African American 
males were more likely than white males to 
report that they played football and basket- 
ball. White students were more likely to 
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In What Regions of the Country 

Are Powerhouse High Schools Located? 

(N=141) 



Table 





Northeast 


South 


Midwest 


West 


U.S. High Schools 


17% 


21% 


31% 


31% 


Powerhouse High Schools 


11% 


43% 


21% 


26% 


Football 


10% 


52% 


18% 


20% 


Basketball 


11% 


32% 


30% 


26% 


Baseball 


10% 


45% ! 


12% 


33% 



NAEP categories used to sort states into regions. Data from U.S. Department of Education, National Center 
for Education Statistics, Common Core of Data , “Public Elementary/Secondary School Universe Survey " 
1999-2000. 



Who Attends Powerhouse High Schools? 

(N=141) 



Table 
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% Poverty 


Black 


White 


Hispanic 


Aslan 


U.S. High Schools 


20% 


13% 


72% 


10% 


3% 


Powerhouse 
High Schools 


21% 


28% 


58% 


9% 


5% 


Football 


18% 


26% 


64% 


6% 


3% 


Basketball 


30% 


42% 


40% 


11% 


6% 


Baseball 


13% 


, 


i 73% 


1 10% 1 


5% 



Poverty and race/ethnicity data from U.S. Department of Education, National Center for Education Statistics, 
Common Core of Data, “Public Elementary/Secondary School Universe Survey,” 1999-2000. 



report participating in “other sports,” primar- 
ily baseball and track. 26 

How do the powerhouses perform on tests 
of reading and math? 

How well do dominant sports schools 
perform on state tests of reading and mathe- 
matics? We collected the powerhouses’ most 
recent test scores in reading and math, in 
most cases 2001 test scores, and converted 
the data to z-scores. Z-scores express school 
achievement relative to the average school 
in each state. State means are set at 0.00, 
with a standard deviation of 1.00. 27 In 
other words, schools with a positive z-score 
scored above average and those with a 
negative z-score, below average. 

Schools serve students of vastly 
different backgrounds, of course, and charac- 
teristics such as the percentage of students 
in poverty are known to influence test 
scores. To place schools on a level playing 
field, z-scores within each state were adjusted 



Achievement 
of Powerhouse 
High Schools 

(Means and standard errors 
of z-scores, N=141) 



Table 
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Z-score 

<SE) 


Powerhouse 


+0.05 


High Schools 


(.06) 


Football 


+0.06 




(.10) 


Basketball 


+0.07 




(-12) 


Baseball 


+0.02 



(.09) 



NOTE: Adjustments made for poverty 
and racial composition. 
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Achievement of Powerhouse High Schools 
by Demographic Characteristics 

(Means and standard errors of z-scores, by quartile, N=141) 



Table 
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Q1 


Q2 


Q3 


Q4 




(Low) 






(High) 


% Poverty 


+0.46* 


+0.06 


-0.11 


-0.24 


(.10) 


(.09) 


(.12) 


(.16) 


% Non-white 


+0.34* 


+0.10 


-0.02 


-0.24 


(.11) 


(-11) 


(.12) 


(.16) 



* p < .05, two-tailed test of z-score = 0 

NOTE: Data report national means of z-scores adjusted for poverty and racial composition at the state level. 



Achievement of Powerhouse High Schools 
by Community 

(Means and standard errors of z-scores, N=141) 



Table 
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Urban 


Suburban 


Rural 


-0.01 


+0.14* 


-0.25 


(.13) 


(.07) 


(.15) 



* p < .05, two-tailed test of z-score = 0 

NOTE: Data report national means of z-scores adjusted for poverty and racial composition at the state level. 




using a regression equation, a standard 
statistical technique employed by researchers. 
The analysis controlled for school racial 
composition and the percentage of students 
qualifying for free lunch, an indicator of 
poverty. This treatment allows for each 
school to be compared to other schools in 
the same state serving students of similar 
racial and socioeconomic backgrounds. 28 

It doesn’t hurt to be a powerhouse. 

As displayed in Table 6, the test data do not 
support the idea that dominance in sports 
diminishes a school’s academic achievement. 
The powerhouses’ average z-score is .05, 
the equivalent of the 52nd percentile. Put 
simply, powerhouse high schools score 
about the same as non-powerhouses. Indeed, 
in all three sports, the powerhouse high 
schools score slightly above — but not 
statistically significantly different from — 
state averages in reading and math. The range 
of school scores is quite large, however, 
from -1.80 to 2.45. Is it possible to explain 
why some powerhouse schools get an 
academic boost from their extraordinary 
accomplishments in sports while other 
schools that are equally accomplished in 
sports do not? 
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Who gets a boost? 

We pooled the powerhouses’ z-scores 
nationally and employed regression analysis 
to tackle this question. Three demographic 
variables emerged as significant — the percent 
of students in poverty, the percent of nonwhite 
students, and being located in a suburban 
area. Rather than attempting to tease apart 
these three variables — a task that is extraordi- 
narily difficult and not always enlightening — 
it is useful to consider them as a demographic 
cluster. What do academically high achieving 
powerhouses look like? They are located 
in relatively wealthy neighborhoods and 
serve predominantly white, non-Hispanic 
populations (see Table 7). Table 8 reveals the 
achievement benefit suburban powerhouses 
get from excelling in sports. Urban power- 
houses score no better or worse than schools 
that are not powerhouses and serve similar 
populations. Powerhouses located in rural 
areas achieve slightly below similar non- 
powerhouse schools, but not to a degree that 
is statistically significant. 

The study has limitations. In any analy- 
sis of school achievement, the purpose of 
controlling demographic variables is to filter 
out the effects of non-school factors on stu- 
dent achievement. Otherwise, in measuring 
student learning, one might actually measure 
poverty or the number of students from 
single parent families or a host of other social 
conditions that affect achievement, then 
misconstrue the effect of these conditions as 
the effect of schools on student achievement. 
The variables employed here are rather crude 
estimates. A study with more precise mea- 
sures of student characteristics might uncover 
different findings. In addition, even if 
demographic characteristics are statistically 
controlled through within-state comparisons 
of school achievement — which calculating 
adjusted z-scores should have accomplished — 
“exporting” the z-scores for national compar- 
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isons could introduce other influences that 
skew the results. 29 

We scrutinized the data more closely to 
see if the suburban advantage for powerhouses 
holds up. Table 9 presents a simple count of 
states in which the average z-scores of power- 
houses and non-powerhouses are compared 
(with poverty and racial composition con- 
trolled). The comparisons are broken out by 
urban, suburban, and rural status. In three- 
quarters of the states, sports powerhouses in 
suburban communities score higher on aca- 
demic tests than non-powerhouse suburban 
schools with similar demographic profiles. 
Suburban powerhouses appear to get an extra 
academic boost from excellence in sports. 

Conclusion 

This study provides an important lesson for 
schools. Some high schools have built extraor- 
dinary programs in basketball, football, and 
baseball, with winning traditions that rival 
famous professional franchises. On average, 
such powerhouses do not sacrifice academics 
for athletics. That is good news. Winning at 
basketball can go hand in hand with winning 
at mathematics. However, high schools with 
advantaged socioeconomic circumstances 
are better able than other schools to integrate 
excellence at sports into an ethos of achieve- 
ment that pervades school culture. 

How do they do it? Speculation is nec- 
essary since the current study did not collect 
information that can answer that question. 
Suburban schools may be more likely to adopt 
policies that stress the importance of athletes’ 
academic progress. They also may have more 
resources that allow them to offer special 
assistance to student athletes who struggle 
academically. Parents may also help. Suburban 
parents of athletes may be able to hire tutors 
to shore up academic deficiencies. 

Being able to replicate success on the 
nlaving field in the classroom is a challenge 
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Comparing the Academic Performance of Powerhouses 
and Non-Powerhouses in 24 states 



Table 

9 



Who scores better? 


Urban 


Suburban 


Rural 


Powerhouses 


12 


15 


4 




( 60 %) 


(75%) 


(44%) 


Non-Powerhouses 


8 


5 


5 




(40%) 


(25%) 


(56%) 


NA 


4 


4 


15 



NOTE: NA (not applicable) refers to states that do not have powerhouses in the category. 



for schools in less favorable socioeconomic 
circumstances. It is not that the study discov- 
ered evidence that athletic success detracts 
from academics, but schools in disadvantaged 
communities do not experience the same boost 
in test scores that schools in wealthier areas 
get from fielding powerhouse teams. Compared 
to their suburban peers, promising young 
athletes in rural areas must overcome two 
handicaps. First, they are much less likely 
to attend a school large enough to become 
dominant in team sports. Second, if they 
are fortunate enough to attend a sports 
powerhouse, the school is less likely to be 
academically successful. More research is 
needed to explain the strategies that allow 
some schools to harness excellence in sports 
and carry it over into classrooms. 

Sports are an integral part of high schools. 
Competition in football, basketball, and 
baseball is vitally important to high school 
athletes, their parents, and communities 
across the country. The surveys of foreign 
exchange students and of American students 
who have studied abroad demonstrate 
that the value placed on athletic excellence 
is deeply ingrained in U.S. teen culture. 
Moreover, participating in sports seems 
to provide numerous benefits to student- 
athletes: elevated status in the eyes of peers, 
closer ties to school, a social network of 
watchful, caring adults, greater motivation 
for academic learning, greater likelihood 
of attending college, and higher wages 
as an adult. These are not trivial rewards. 
They underscore the need for schools to 
offer sports as part of an education that 
stresses excellence in all of its dimensions. 
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C HARTER SCHOOLS ARE PUBLIC SCHOOLS OF CHOICE. 
Parents enroll their children in a charter school because they 
want their kids to attend that particular school, not, as in the 
case of regular public schools, because the family’s home sits within the 
school’s enrollment boundaries. In exchange for meeting educational 
outcomes, charter schools are freed from most regulations. The outcomes 
are promised in a renewable license to operate a “charter,” which is 
reviewed periodically. In the typical state, a review is conducted every 
five years. Schools that meet their goals are granted a fresh charter. 

Those that don’t may be closed down. 



The nations first charter school legislation 
was passed in Minnesota in 1 99 1 , and a few 
schools opened their doors the following fall. 
California followed in 1992, and six more 
states in 1993. Then the charter movement 
spread like wildfire. In 1995, eighteen states 
had passed charter laws. By 1999, the number 
of charter states reached thirty-seven. Supported 
by leading Democrats and Republicans, 
charter schools grew in a single decade from 
a modest experiment in educational reform 
to 2,400 schools open in the 2001-2002 
school year. 30 

How do charter schools perform on tests 
of academic achievement? Two major studies 
recently reviewed the existing research on 

0 
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this question, one released in 2001 by Gary 
Miron and Christopher Nelson, the other in 
2002 by Brian Gill and colleagues at RAND. 
Both reviews described the evidence on char- 
ter school achievement as mixed to slightly 
favorable. The reports also stressed that, 
with so many charter schools being relatively 
new, much more research into charter school 
achievement must be conducted to arrive 
at any definitive conclusions about their 
academic performance. 31 

A study of charter schools 

Brown Center researchers selected a sample 
of charter schools for the purpose of examin- 
ing their academic achievement. The specific 
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Summary Statistics for Charter School Sample 

(N=376) 



Table 



10 





Mean 


Standard 

Deviation 


Minimum 


Maximum 


Enrollment 


252* 


351 


18 


2,938 


Poverty 


0.38 


0.25 


0.00 


1.00 


White 


0.54 


0.36 


0.00 


1.00 


Black 


0.23 


0.33 


0.00 


1.00 


Hispanic 


0.18 


0.24 


0.00 


0.99 


Aslan 


0.03 


0.08 


0.00 


0.81 


At-Risk 


0.13 


0.34 


0 


1 


New In 1999 


0.25 


0.43 


0 


1 



* For Enrollment, the median is substituted for the mean (368). At-Risk denotes schools specially designed to 
serve at-risk students. 



research objective was to compare charters’ 
performance on state tests from 1999 to 
2001 with the performance of regular public 
schools. Did they score better, worse, or 
about the same? In addition to test scores, 
information was collected on the number 
of students enrolled in charter schools and 
the schools’ racial and socioeconomic com- 
position. Demographic data were used to 
compare charters to regular public schools 
serving students with similar background 
characteristics. 

A limitation of the study is important 
to note. With the data at hand, it is impossi- 
ble to tell whether charter schools’ test scores 
reflect the quality of education at the schools. 
This is because charters’ test scores may be 
influenced by what statisticians call “selec- 
tion effects.” Students attend charters because 
their parents have decided to send them 
there. They select the school. If charter 
students or their families are fundamentally 
different from kids attending regular public 
schools, these differences — not differences 
in the quality of schools — may produce differ- 
ences in school test scores. Selection effects 
can be negative or positive. Charter students 
may have struggled academically before par- 
ents placed them at the school. But they also 
are probably blessed with parents who take 
an active interest in their children’s education. 

Despite this caveat, examining the test 
scores of charter schools is useful. States are 
currently testing charter school students, 
releasing the test scores, and issuing watch 
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lists comprised of schools, including charters, 
that are failing. This study aggregates charter 
schools’ scores across ten states and looks 
for patterns in their performance. Explaining 
why these patterns occur will require more 
specific information on the children attending 
charters and more rigorous statistical tech- 
niques than employed here. 

We focused on ten states with the fol- 
lowing qualifications: they had at least thirty 
charter schools open in 1999, tested students 
in grades 4,8, and 10 (allowing for substitu- 
tion of adjacent grade levels), and used the 
same achievement test in 1999, 2000, and 
2001. The ten states had a total of 638 charter 
schools operating in 1999. Of these schools, 
we were able to assemble a complete panel 
of data on 376 charters. We did not add char- 
ters opening in 2000 or 2001 to the sample 
so that the number of schools remains fixed 
during the three year time frame. Test data 
were collected — on charters and regular 
public schools — from state departments of 
education and from websites maintained 
by state assessment programs. Demographic 
and enrollment statistics were taken from 
the National Center of Educational Statistics 
Common Core of Data. 32 

Table 10 contains summary statistics 
on the study’s sample of 376 charter schools. 
The average charter in the study is noticeably 
smaller than regular public schools in the U.S. 
(the national average is 520 students for a reg- 
ular public school). 33 Median enrollment in 
the charters is 252. About 38% of students 
in the charters are poor, close to the national 
average for regular public schools. Approxi- 
mately 13% of the charters are specially 
designed to serve at-risk students. The charters 
serve a higher proportion of black students 
(23% vs. 17% nationally) and Hispanic 
students (18% vs. 15% nationally) than the 
average public school. About 25% of the 
charters in the study were new in 1999. 34 
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Estimating charter school 
performance 

To estimate the charters’ academic performance, 
we employed a strategy similar to that used 
with the sports powerhouses in Part II. We 
collected test score data from the ten states, 
combined reading and math achievement 
from 1999 through 2001 into a composite 
score, and computed z-scores for all schools — 
regular and charter — in each state. The 
z-scores in Table 1 1 have been statistically 
adjusted for student background (socioeco- 
nomic status and racial composition) and 
weighted for enrollment. This allows us to 
compare charter schools to regular public 
schools with similar demographic character- 
istics. Within each state, an average z-score 
is 0.00. 35 

Computing a weighted mean allows 
larger schools to count for more than smaller 
schools. Why is this done? Imagine that you 
want to compute an overall test score for the 
schools in a small town. The town has two 
schools. How should the town’s average 
achievement be reported? Pretty simple, one 
might answer, add the two schools’ scores 
together and divide by two. But what if one 
of the schools has 300 students and the other 
only thirty? Treating the schools the same 
would be misleading. Giving the first school 
ten times more weight in computing the aver- 
age score provides a better indication of how 
students are doing in the town as a whole. It 
is also a statistical property of test scores that 
those from large schools are more reliable 
than those from small schools, another reason 
for weighting averages by enrollment. Recall 
that many of the charter schools in the cur- 
rent study are quite small. 

Charter schools score significantly 
below regular public schools on achievement 
tests, about .24 z-scores below average. As 
mentioned above, care must be exercised 
in interpreting the charter school test score 



deficit. The study does not possess evidence 
and therefore cannot shed light on why 
charters score below average. One possible 
explanation is that charter schools are not 
doing a very good job. But an equally plausi- 
ble explanation is that charters attract large 
numbers of students who are struggling 
academically in public schools before ever 
setting foot on a charter school campus. 

The charters, in fact, may be doing an excel- 
lent job, bringing these low achievers up 
to a level that, although still below average, 
is not as low as when the students attended 
public schools. 

Data on students’ achievement before 
and after they enrolled in charters would 
help determine which of these explanations 
is true. Computing the academic gains that 
students make as they move through grade 
levels is better at isolating a school’s contribu- 
tion to learning than simply analyzing the 
level at which a school is performing. In addi- 
tion, an analysis of how students perform 
after being randomly assigned to charter and 
regular schools would be valuable. Comparing 
the achievement gains of randomized samples, 
for example, has significantly advanced the 
research on vouchers. Many charter schools 
are oversubscribed, with more people seek- 
ing entry than space allows. Charter school 
administrators usually decide who gets to 
attend by a random draw, making such exper- 
imental research possible. 36 

By itself, a simple test score is not the 
best tool for explaining a school’s contribu- 
tion to its students’ learning. But it remains 
the best indicator for assessing what students 
at any particular school know and can do in 
academic subjects — whether they can read, 
compute accurately, grasp the fundamental 
principles of science, or understand the impor- 
tance of famous events in history. As Table 
1 1 shows, charter schools in four states 
(Massachusetts, Michigan, Minnesota, Texas) 



Charter School 
Achievement 
in Ten States 

{Means and standard errors 
of z-scores) 



Table 

ii 



State 


Z-score 


Arizona 


-0.03 


(N=51) 


(.11) 


California 


-0.02 


(N= 97) 


(.07) 


Colorado 


+0.18 


(N=31) 


(.12) 


Florida 


-0.37 


(N=29) 


(.22) 


Massachusetts 


-0.S3* 


(N=21) 


(.16) 


Michigan 


-0.63* 


(N= 84) 


(.08) 


Minnesota 


-0.44* 


(N=16) 


(.16) 


Pennsylvania 


+0.0S 


(N=ll) 


(.27) 


Texas 


-1.09* 


{N= 25) 


(.33) 


Wisconsin 


-0.18 


(N= 11) 


(.41) 


Average 

(N=376) 


-0.24* 

(.04) 



* p < .05, two-tailed test of z-score = 0 

NOTE: Adjustments made for poverty 
and racial composition, weighted 
by enrollment. 



Achievement of 
Charter Schools 
by Community 



Table 

12 



{Means and standard errors 



of z-scores, N 


=376) 




Urban 


Suburban 


Rural 


-0.13 


-0.34* 


-0.36* 


(.07) 


(.06) 


(■Hi 



* p < .05, two-tailed test of z-score = 0 

NOTE: Adjustments made for poverty 
and racial composition, weighted 
by enrollment. 
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Achievement of 
Charter Schools 
by Enrollment 

(Means and standard errors 
of z-scores, N=376) 



Table 

Ti3 



Up to 155 


-0.44* 


Students 


(.11) 


156-246 


-0.36* 


Students 


(.11) 


247-480 


-0.24* 


Students 


(.10) 


481-2,938 


-0.19* 


Students 


(.08) 



p < .05, two-tailed test of z-score = 0 

NOTE: Adjustments made for poverty 
and racial composition, weighted 
by enrollment. 




score significantly below state averages for 
similar schools. Charter achievement in the 
other six states is indistinguishable from 
average. Colorado charters’ raw scores are 
significantly above average (+.44); however, 
once school composition has been controlled, 
the Colorado charters’ performance is not 
significantly different from the performance 
of schools serving similar populations (+.18). 
It is important to note that, compared to 
other states, Colorado’s charters are uniquely 
suburban. 

Urban charters score higher than 
suburban or rural charters. 

When charters in the study are grouped 
together by common characteristics and 
compared across states, several interesting 
patterns emerge. For example, urban charter 
schools exhibit higher achievement than 
suburban or rural charters (see Table 12). 
Urban charters’ test scores are just slightly 
below state averages (-. 13). Rural and 
suburban charters, on the other hand, score 
significantly below schools with similar 
racial and socioeconomic profiles. This is 
important when thinking about charters as 
somewhat risky educational options for 
parents. Charters may be housed in old strip 
malls, church basements, or long-abandoned 
school buildings. They often hire teachers 
without official teaching certificates or exten- 
sive experience in the classrooms. And, as 
pointed out above, the research on charters’ 
effectiveness is inconclusive. The results 
in Table 12 suggest that parents in urban 
schools are assuming less risk — at least when 
it comes to a prospective school’s academic 
standing — when transferring children to 
charters than parents who do the same in 
rural and suburban areas. This is good news 
since urban parents may be leaving relatively 
bad schools and therefore be willing to 
consider riskier alternatives. 

36 



Large charters score higher than 
small charters. 

Table 13 examines achievement in charters 
of different sizes. Large charters achieve at 
higher levels than smaller charters. This is 
somewhat counter-intuitive considering the 
popular “small school” movement, which 
advocates breaking down large schools into 
smaller organizational units. Small school 
advocates are concerned that students get 
overlooked in large institutions and are more 
likely to bond with an adult in smaller set- 
tings. But the largest charters in this study, 
those serving 481 students or more, are 
achieving significantly higher on state tests 
(-.19) than charters with 155 or fewer 
students (-.44). 

What could explain the discrepancy? 
One potential reason is related to school admin- 
istration. Many large charters are run by 
educational management organizations, or 
EMOs. Professionals might have the know-how 
to produce higher achievement compared to 
the typical mom and pop administration of 
small charter schools. Economies of scale may 
also play a part. Even the most devoted small 
school advocates recognize that schools might 
have to grow to a certain size to be viable as 
institutions. Extremely small schools may 
have trouble raising funding or securing loans, 
receiving favorable treatment from vendors 
when purchasing textbooks and other instruc- 
tional materials, or attracting star teachers. Some 
charters may be too small to operate efficiently. 

New charters have depressed scores 
for the first two years. 

In 1999, about one-fourth of the study’s 
charter schools were new. Their test scores 
fluctuate in an interesting way. In their first 
and second years, the new schools scored 
significantly below schools that were already 
open in 1999 (see Figure 13). In the third 
year, 200 1 , the new charter schools caught 
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up with the older ones. Previous state evalua- 
tions of charter schools have noted that test 
scores are depressed the first year that a new 
charter is open. The current study is indicat- 
ing that the negative new school effect extends 
for two years. 

What is behind the two year lull in new 
charters’ test scores? Speculation is necessary 
here. The stress and strain of opening a new 
school may be partially to blame. Founders 
of charter schools face a mountain of difficult 
tasks — finding adequate facilities, hiring 
teachers, preparing curriculum. Moreover, 
students are moving from their previous 
schools. Prior research has shown quite con- 
vincingly that student mobility depresses 
test scores. 37 The new school effect could 
also mean that right from the start charter 
schools are attracting a disproportionate 
share of low achieving students. This relates 
to the discussion of selection effects above. 



If charters attract initially low achieving 
students, it may take two years for these 
students to be brought up to speed. The new 
schools in 1999 progressed from -.44 to 
-.21, a gain of .23 z-scores. Again, careful 
analysis of longitudinal data from students 
would shed light on whether the new school 
phenomenon is real or a statistical aberration. 

Summary and recommendations 

This section of the report presented a study 
of charter schools test scores from 1999 to 
2001. In the study’s ten states, charters scored 
about .24 z-scores below regular public 
schools of similar composition. Urban char- 
ters scored higher than suburban or rural 
charters, larger charters scored higher than 
smaller charters, and charters already in exis- 
tence in 1999 scored higher than charters 
opening their doors for the first time that 
year. New charters’ test scores lagged existing 
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It is clear that charter 
schools are scoring below 
average on tests of 
academic achievement, 
but why they do so 
remains a question. 



charters for two years, but in 2001, the new 
charters’ scores jumped, and the test scores 
from new and old charter schools were statis- 
tically indistinct. 

Two recommendations are appropriate. 
First, additional study of charter schools is 
needed, especially research that controls for 
selection effects. It is clear that charter schools 
are scoring below average on tests of academic 
achievement, but why they do so remains a 
question. It could be because charters offer 
an inferior education, or it could be because 
charters attract students who are low achiev- 
ing in the first place. 

The second recommendation addresses 
the new school effect. If charter schools are 
at a systematic disadvantage for the first two 
years that they are open, policymakers should 
consider special treatment for them in account- 
ability programs. States frequently place 
schools with three years of poor test scores 
on watch lists or warning lists. Sanctions are 
threatened if achievement does not improve. 
A grace period might be appropriate for new 
charters — and for new regular public schools 
if they experience the same phenomenon — 
so that they are given a fair chance to produce 
learning. States should consider delaying the 
“accountability clock” on new schools until 
the third year. 

A final word on charter schools. There 
may be no such thing as a “charter school 
effect” in the sense that an inherent quality 



of charter schooling influences achievement. 
Charters are nothing more than an institu- 
tional vessel into which several elements 
are poured — a founder’s inspiration, a new 
principal and teaching staff, a new curricu- 
lum, and perhaps several innovative ideas. 
Charters are incredibly diverse. There are 
Montessori charters, Waldorf charters, back- 
to-basics charters, Afrocentric charters, 
and Core Knowledge charters. Some charter 
schools serve gifted youngsters, and others 
serve adolescents recently released from the 
criminal justice system. 

Charters share two characteristics: 
they serve students whose families have 
chosen for them to be there, and they commit 
to attaining certain outcomes within a stipu- 
lated period of time. These are elements of 
governance. And they are process variables. 

If future research shows that charters pro- 
duce a universal educational gain or loss 
from such a diverse group of schools, then 
how schools are governed will be proven 
more influential than all but a few people 
have ever imagined. The greater likelihood 
is that charters will be found to produce a 
wide range of outcomes. Some charters will 
be terrific places for educating children 
and others will be failures. Identifying the 
characteristics of excellent charter schools 
and encouraging their adoption should 
be the main objectives of the next wave of 
charter school research and policy. 
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